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Abstract: This study models the registration process in order to directly examine 
the possibility of pre-existing differences between undergraduate students who 
choose to take sendee-learning classes compared to those who do not. Foremost 
among the findings, on most measures there was no significant effect of self- 
selection. However, students who chose a hypothetical course with a sendee- 
learning component reported higher civic responsibility and civic action scores, 
and reported lower socioeconomic status and greater enjoyment of past sendee 
learning than those who chose the hypothetical course without the sendee- 
learning component. Challenges and opportunities related to random assignment 
within sendee-learning research are discussed. 

Keywords: service learning, self-selection, civic action, civic responsibility, 
learning outcomes. 

The educational, civic and personal benefits of service-learning have been richly noted in an 
ever-expanding scholarly literature. Previous research has indicated that service-learning is 
related to a number of social, academic, citizenship and personal outcomes that are beneficial to 
students (for comprehensive reviews see Conway, Amel, & Gerwien, 2009 and Eyler & Giles, 
1999). For example, students engaged in service-learning show increased civic engagement, 
cognitive development, leadership skills, and work better in groups (Bringle, 2005; Wells & 
Grabert, 2004; Hollander & Meeropol, 2006). Compared to students enrolled in traditional 
courses, students who took part in service-learning classes showed increased grade point 
averages, gains in knowledge, and degree aspirations (Conway et ah, 2009). Perhaps most 
impressive, a common theme in the literature is that the positive effects of service-learning are 
bi-directional; they flow from the student to the community and back to the student (Kraft, 
1997). 

While this research is compelling, another growing body of literature has discussed 
methodological flaws and limitations in service-learning research (e.g. Shumer, 2000; Bringle & 
Hatcher, 2000; Steinberg, Bringle, & Williams, 2010). For example, Steinberg and colleagues 
(2010) note several methodological challenges in designing a rigorous study of service-learning 
outcomes. Some of these challenges include (but are not limited to) lack of controls or 
comparison groups and absence of random assignment. These particular limitations leave 
researchers unable to infer a causal relationship between the measured outcome and participation 
in service-learning. Without random assignment to conditions (service-learning and at least one 
control group), any number of unmeasured variables could account for improved outcomes, and 
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researchers cannot be confident that it was the service-learning experience itself that resulted in 
change. 

For example, a study that examined reflection writings across a service-learning class 
found that students produced increased quality of thought, higher order reasoning skills, and 
improved critical thinking in their writing (Ash, Clayton, & Atkinson, 2005). However, a control 
group (traditional class) was not included in the study to see if these benefits are merely artifacts 
of frequent student writing and reflection. Without this control group, it is impossible to truly 
infer that the gains in writing were due to participation in service-learning (see also Bringle & 
Hatcher, 2000). Similarly, examining a difference at a single time-point in a class is problematic. 
Gallini and Moely (2003) found differences between service-learners and students not enrolled in 
service-learning in an end of semester survey of engagement and retention. Without a pre-test, 
however, limited conclusions may be drawn about this data because the two groups of students 
may have had pre-existing differences on measured and unmeasured variables. 

Another looming confound is the opportunity for students to self-select into service¬ 
learning or traditional classes (Steinberg et al., 2010). The issue of self-selection bias is, perhaps, 
the most pervasive methodological issue in the service-learning literature. In most institutions, 
students may simply opt-out of taking service-learning courses (unless there is a service-learning 
or community service learning graduation requirement). Consequently, students enrolled in 
service-learning courses may be different in important ways from students who are not enrolled 
in service-learning courses. Interestingly, many of the variables that are often reported to have 
changed because of service-learning are different even prior to the service-learning experience 
(e.g., Eyler, Giles, & Braxton, 1997; Gallini & Moely, 2003). For example, students who 
participate in service-learning as undergraduates are reported to have previous experience with 
volunteerism in their community, have high self-rated leadership ability, and to report a 
commitment to participating in community action programs (Sax & Astin, 1997). Essentially, 
students that are community and service focused self-select into service-learning courses. Thus, 
gains in similar measures after participating in service-learning may be indicative of the 
students’ preference for service-learning or their receptiveness to the service-learning 
experience, rather than to the service-learning experience per se. This non-random assignment of 
participants also limits the degree to which conclusions about the efficacy of service-learning 
generalize from students who opt-in to service-learning to students who may opt-out or who are 
required to complete service-learning (see also Bringle & Hatcher, 2000). 

Random assignment requires that chance alone determines whether a student is placed in 
a service-learning or control class. This is exceedingly difficult in educational settings where 
student choice is often highly valued. There is a sparse literature that attempts to randomly assign 
learners to traditional and service-learning sections of the same course. Typically, the sections 
that are service-learning are not marked for students at the time of registration, thus students 
theoretically select a course that contains service-learning seemingly at random (e. g. Markus, 
Howard, & King, 1993; Osborne, Hammerich, & Hensley, 1998). These studies typically find 
significant effects of service-learning on variables as diverse as self-esteem, cognitive 
complexity, self-concept, application of principles to novel situations, greater awareness of 
societal problems, grades, and learning. However, the fact that students select into particular 
sections at registration even without knowledge of whether or not there will be a service-learning 
component opens the possibility that other extraneous variables may impact these experiments 
with seemingly random assignment. For example, time of day, perception of assigned instructor, 
and even whether or not a friend is also registering for the same section may influence the 
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outcome variables. In fact, the number of extraneous variables that could influence section 
choice is infinite. 

Two popular ways to address these problematic methodological issues are by employing 
a longitudinal design, and/or by measuring and then statistically controlling for pre-existing 
differences between groups. In regard to longitudinal design, Astin, Vogelgesang, Ikeda, and 
Yee (2000) followed students over a four year period to examine change in multiple service¬ 
learning related outcomes. Evaluating change over time strengthens conclusions about the 
possible impact of service on learning, but does not directly address possible self-selection 
effects. 

Another popular way to address the possibility of self-selection is to statistically control 
for pre-existing differences (Astin et ah, 2000). For example, if two classes had different starting 
scores on self-efficacy, this variable could be analyzed as a covariate allowing a researcher to 
compare end of term differences in self-efficacy while controlling for the pre-existing 
differences. Unfortunately, these statistical techniques can only control for variables that the 
researchers have already identified as potentially influential factors (Bringle & Hatcher, 2000; 
Steinberg, et ah, 2010). Even statistically controlling for some variables may not fully address 
questions of self-selection—students that self-select into a service-learning class may show 
greater gains than non-service-learners because of an infinite number of unmeasured variables. 
Though unlikely, it is possible for any number of extraneous variables to influence outcome 
(e.g., degree of extroversion). While some compelling studies have attempted to statistically to 
control for initial differences between groups, the studies are limited in their ability to assert 
service-learning as the sole cause for their results because of the absence of true random 
assignment (e.g., Bringle & Kremer, 1993; Eyler et ah, 1997; Gallini & Moely, 2003). 

The goal of this research is to quantify the effects of service-learning self-selection. 
Because the structure of most academic settings precludes true random assignment of students to 
courses, we chose to model the registration process and then evaluate the effects of self¬ 
selection. In the current study, students were presented with the option of two classes that 
differed only by their inclusion of service-learning, and were asked to make a choice. Students 
who chose the course with the service-learning component were then compared to students who 
chose the traditional course on demographics and several commonly used outcome measures in 
the service-learning literature. Our hope is that this data will inform future research (and offer 
perspective on past research) as to the effects of service-learning self-selection. 

Method 

Participants 

Undergraduate students (N = 141) were recruited from general education psychology classes at a 
Midwestern public liberal arts university. Participant age ranged from 17 to 51 (M= 21.0, SD = 
4.4), and year in college ranged from 1 to 6 (M = 2.4, SD = .47). Self-reported ethnic identity 
was as follows: 88% European American, 4% multiethnic, 3% another preferred identity, 2% 
Native American, 1% African American, 1% Asian American, and 1% Latino. 

Procedure 
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Participants completed all measures described below after giving informed consent. They were 
provided the following prompt, “Both of the classes listed below will require the same amount of 
in-class and outside of class work. If you were required to register for one of these courses, 
which would you choose?” and the following response options, “a general education class with a 
service learning component” or “a general education class without a service learning 
component.” The question about class choice was always completed first, and the demographic 
questionnaire was always completed last, but the order of the measures described below was 
counterbalanced to offset possible order effects. 

Measures 

Demographic questionnaire. Participants were asked to report their age, year in school, 
sex, sexual orientation, race/ethnicity, past experiences with service learning and/or community 
service, and socioeconomic status. 

Higher Education Sendee-Learning Survey (HES-LS). The HES-LS (Furco, 2000) is a 
29-item questionnaire developed to assess four service-learning constructs: civic responsibility, 
academic attitude, career development, and empowerment. Participants responded using a 4- 
point Likert scale, with higher scores indicating greater endorsement of the constructs. Obtained 
alphas for the current sample are reported in Table 1. 

Civil Attitudes and Skills Questionnaire (CASQ). The CASQ (Moely et ah, 2002) is a 45- 
item questionnaire comprised of six subscales designed to assess student attitudes regarding civic 
action, interpersonal and problem-solving skills, political awareness, leadership skills, social 
justice, and diversity. Participants responded suing a 5-point Likert scale, with higher scores 
indicating greater endorsement of the attitudes. Obtained alphas for the current sample are 
reported in Table 1. Differences in leadership skills were not examined due to low reliability of 
this scale in the current sample. 

The Self-Efficacy Scale. The Self-Efficacy Scale (Sherer et ah, 1982) is a 32-item 
questionnaire with a Likert scale from 1 (Disagree Strongly) to 6(Strongly Agree). Self-efficacy 
is a person’s belief that the outcome of one’s behavior will be successful. Obtained alphas for the 
current sample are reported in Table 1. 

Results 

Self-selection effects were examined with independent samples /-tests. On most measures, no 
significant effect of self-selection was found (see Table 1). However, results indicated that 
course choice had a statistically significant effect on civic responsibility /(139) = 3.93, p < .01, 
civic action /(139) = 3.62, p < .01, socioeconomic status /(138) = -2.27, p = .02, and enjoyment 
of past service-learning till) = 4.00, p <.01. On average, students who chose the hypothetical 
course with the service-learning component reported higher civic responsibility (M = 2.87, SD = 
.33) and civic action (M= 3.70, SD = .69) than those who chose the hypothetical course without 
the service-learning component (M= 2.63, SD = .38, and M = 3.24, SD = .74, respectively). On 
average, students who chose the hypothetical course with the service-learning component 
reported lower socioeconomic status (M = 3.71, SD = 1.15) and greater enjoyment of past 
service-learning (M= 3.89, SD = 1.01) than those who chose the hypothetical course without the 
service-learning component (M= 4.15, SD = .95, and M= 2.86, SD = .74, respectively). 
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Table 1. Differences by class choice 


Class Choice 
With Service Without 
Learning Service 
Learning 


Variable 

M (S.D.) 

M (S.D.) 

t 

df 

P 

a 

General Efficacy 

4.47(.61) 

4.45(.73) 

.21 

139 

.84 

.87 

Social Efficacy 

4.12(.79) 

3.95(1.01) 

1.10 

139 

.27 

.75 

Academic Attitude 

3.22(.34) 

3.11(.31) 

1.76 

139 

.08 

.65 

Civic Responsibility 

2.87(.33) 

2.63(.38) 

3.93 

139 

<.001 

.83 

Career Development 

3.11 (.46) 

3.12(.57) 

-.125 

139 

.90 

.77 

Empowerment 

3.05(.33) 

3.15(.42) 

-1.54 

139 

.13 

.69 

Civic Action 

3.70(.69) 

3.24(.74) 

3.62 

139 

<.001 

.91 

Skills 

4.09(.54) 

4.21(.46) 

-1.27 

139 

.21 

.85 

Political Awareness 

2.85(.76) 

2.94(.82) 

-.66 

139 

.51 

.85 

Leadership Skills 

3.47(.43) 

3.47(.43) 

NA 

NA 

NA 

.09 

Social Justice Attitudes 

3.69(.53) 

3.60(.61) 

.814 

139 

.42 

.69 

Diversity Attitudes 

3.76(.64) 

3.71(.67) 

.46 

139 

.65 

.70 

Age 

20.69(4.22) 

21.58(4.78) 

-1.13 

137 

.26 

NA 

Year in College 

2.32(1.20) 

2.57(1.32) 

-1.17 

137 

.24 

NA 

SES 

3.71(1.15) 

4.15(.95) 

-2.27 

138 

.03 

NA 

Enjoy Past SL 

3.89(1.01) 

2.86(.96) 

4.00 

72 

<.01 

NA 


Notes. Skills - Interpersonal & Problem Solving Skills. Enjoy Past SL - Enjoyment of Past 
Service Learning. NA - not applicable. The low alpha value for the Leadership Skills 
scale precluded further analysis, and we caution researchers using this measure to check 
their sample alpha value prior to data analysis/interpretation. 

Discussion 

This research modeled the registration process to examine potential differences between students 
who chose to engage in service-learning and students who chose traditional courses. By 
modeling this process, we were able to hold everything about the two courses constant with the 
exception of the presence or absence or service-learning, something that is rarely possible in 
applied academic settings. Use of this novel methodology is a significant contribution to the 
research literature, particularly in the scholarship of teaching and learning where random 
assignment is often impractical. 

Another notable finding of the current research was that the two groups of students 
shared more similarities than differences in commonly used service-learning outcome measures. 
This is a significant finding because it ameliorates concerns about self-selection bias. 
Differences, outlined below, were small in comparison to the overall finding of similarity 
between groups. 

Despite the finding of overall similarity, students in the current study who chose service¬ 
learning had lower socioeconomic status, higher sense of civic responsibility and action, and 
greater enjoyment of past service experience compared to students who chose the traditional 
course. In particular, it is important to note the differences in civic action and responsibility 
because these attributes have been reported to increase after engaging in service-learning 
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(Moely, McFarland, Miron, Mercer, & Illustre, 2002; Moely, Furco, & Reed, 2008). The finding 
that students of lower socioeconomic status were more likely to choose the service-learning class 
is a unique contribution to the literature. Previous research has found no difference in 
volunteerism in college students as related to socioeconomic status (Walpole, 2003). Another 
study found that students of low socioeconomic status and first-generation college students spent 
less time volunteering than their peers (Pascarella, Pierson, Wolniak, & Terenzini, 2004). In 
adults, higher socioeconomic status is positively correlated with amount of volunteerism 
(Wilson, 2000). The current results may offer a new developmental finding regarding college 
students of low socioeconomic status and community service, and deserves additional attention 
in future research. 

Although the current results are fascinating, some limitations must be noted. Our attempt 
to model the registration process was limited by participants’ knowledge that their choice of 
course would not result in having to actually take the course. The choice students made in this 
hypothetical situation may or may not directly translate into what they would choose in a real 
registration process. Our sample was also primarily white and limited to one Midwestern 
university, and conclusions would be strengthened by replication in more diverse samples. 

Applied research is inherently complicated. For example, while random assignment to 
service-learning is the most methodologically sound option, this is not always feasible— 
particularly if other variables cannot also be controlled (i.e., instructor, course, time of offering, 
etc). All research designs have pros and cons, and the variety of options available allow the 
strengths of one design to offset the weaknesses of another. In addition to the modeling approach 
outlined in the current study, other applied disciplines offer potential options for creative 
research design (cf. clinical psychology, Kazdin, 2002). 

One potentially interesting way to approach a control group is to compare students 
enrolled in a class with service-learning, and students who were randomly wait-listed for the 
same course (see Seider, Gillmor, & Rabinowicz, 2010). This option ensures that students who 
have similar dispositional or attitudinal characteristics based on their desire to engage in service¬ 
learning are involved. If outcomes vary between students enrolled in the course and students on 
the wait-list, the additional benefits may be related to service-learning. One limitation is that this 
design does not rule out the fact that changes may occur from exposure to the academic material 
and not the service-learning itself. This design may also be problematic because of large 
differences in sample-sizes between students enrolled in a class and those wait-listed. In practice, 
this design is often limited by higher levels of attrition in the wait-listed group. This type of 
design, however, is another clever and innovative option for examining the effects of service¬ 
learning on student outcomes. 

Some educational institutions require service-learning as a graduation requirement. This 
type of requirement may assist researchers in establishing appropriate control groups. If all 
students are mandated to complete a service-learning requirement, the wait-list control design 
becomes quite feasible. However, previous work has demonstrated that mandatory volunteerism 
has a negative impact on student outcomes (e.g. Stukas, Snyder, & Clary, 1999). Because of the 
differences in students who would willingly enroll in a service-learning class, instructors and 
researchers need to be cautious in drawing conclusions about service-learning from “mandatory” 
service-learning experiences. Students enrolled in a mandatory service-learning class may show 
null results or smaller effect sizes because the effect of service-learning may be attenuated by the 
presence of unwilling students. 
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The current study indicated that students who chose a service-learning class had higher 
civic responsibility and civic action, lower socioeconomic status, and greater enjoyment of past 
service-learning than students who chose the same class without a service-learning component. 
The current research also indicated that the two groups of students were similar in regard to 
general efficacy, social efficacy, academic attitude, career development, empowerment, skills, 
political awareness, leadership skills, social justice attitudes, diversity attitudes, age, and year in 
college. This is knowledge that has implications for how future researchers contextualize and 
understand their findings. For example, in the absence of random assignment and control groups, 
a researcher could have more confidence in results indicating higher social justice attitudes than 
in results indicating higher civic responsibility. 

As a final thought, the methodological issues inherent in service-learning research do not 
undennine the value of this scholarship or this pedagogy. Our hope is to draw awareness to 
potential methodological issues that greatly impact the potential causal inferences that may be 
drawn from service-learning research. As discussed in the introduction, many methodological 
issues complicate the service-learning literature. In particular, this study highlights the need to 
consider the possible impact of self-selection, and to control for it when possible. This need is 
especially great when the study will examine measures that the current results suggest vary 
among students engaging (or not) in service-learning. Though challenges exist with random 
assignment within the service-learning literature, the most important finding of the current study 
is that students who self-select into service-learning are mostly similar to their peers who do not. 
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